Solution of Linear Programming and Non-Linear Regression Problems Using Linear M-Estimation Methods

This Ph.D. thesis is devoted to algorithms for two optimization problems, and their implementation. The algorithms are based on solving linear M-estimation problems. First, an algorithm for the non-linear M-estimation problem is considered. The main idea of the algorithm is to linearize the residual function in each iteration and thus calculate the iteration step by solving a linear M-estimation problem. A 2-norm bound on the variables restricts the step size, to guarantee convergence. The other algorithm solves the linear programming problem. If a variable in the primal problem has both a lower and an upper bound, it gives rise to an edge in the dual objective function. This edge is “smoothed” by replacing it and its neighbourhood with a quadratic function, thus making it possible to solve the approximated problem with Newton’s method. For variables with only lower or only upper bounds, a quadratic penalty function is used on the dual problem. In this way also variables with one-sided bounds can be handled. A crucial property of the algorithm is that once the right active column set for the optimal solution is identified, the optimal solution is found in one step. The implementation uses sparse matrix techniques. Since it is an active set method, it is possible to reuse the old factor when calculating the new step. This is accomplished by upand downdating the old factor, thus saving much computation time. It is only occasionally, when the downdating fails, that the factor instead has to be found with a sparse multifrontal LQ-factorization.